6.962: Week 4 Summary of Discussion Topic: Codes on Graphs 1 a Brief History of Codes on Graphs

1962 Gallager invents low density parity-check LDPC codes 1112 and what is now known as the sum-product iterative decoding algorithm for a p osteriori probability APP decoding. 1981 Tanner founds the eld of codes on graphs, introducing the bipartite graphical model now widely popular referred to as a Tanner graph" and the decoding algorithm now called the min-sum" or max-product" algorithm. Tanner generalizes the single parity-check local constraints of LDPC codes to arbitrary linear code constraints. 1993 Turbo codes are invented, and the gap to capacity is essentially closed. However, the decoding algorithm remains to be analyzed, and the subsequent development of turbo codes occurs largely independently of codes on graphs. 1994-1996 Spielman's thesis introduces LDPC codes based on expander graphs 5556, which have linear-time encoding and decoding algorithms but still decent performance. These results are quickly adapted by Alon and Luby 77 to construct low-complexity, near-optimal coding schemes, the so-called tornado codes," for reconstructing large Internet les in the presence of packet erasures. Meanwhile, MacKay and Neal use simulations to show that long LDPC codes can perform as well as turbo codes 8889, shocking the coding world. Independently, Wiberg produces his thesis Codes and decoding on general graphs" 100, which Prof. Forney calls a triumph of uniication that established the intellectual foundations of the eld." In his thesis, Wiberg extends Tanner graphs to include state variables as well as symbol variables, permitting uniication of turbo codes, LDPC codes, and trellis codes together in a common framework. 19988present Kschischang, Frey, and Loeliger generalize Wiberg-type graphs and introduce factor graphs" 111, natural graphical models of a global function that can be factored into a product of local functions. Forney introduces normal graphs" 12, deened as Wiberg-type graphs in which all symbol variables have degree 1 and state variables have degree 2, leading to a clean separation of functions in sum-product decoding. Divsalar, Jin, and McEliece introduce repeat-accumulate" RA codes 1333simple turbo-like" codes combine simple repetition codes, a 2-state rate-111 convolutional 1